data.table dplyr flextable ggplot2 hashTagR hms knitr
TRUE TRUE TRUE TRUE TRUE TRUE TRUE
kableExtra lubridate readr rtweet tidytext wordcloud
TRUE TRUE TRUE TRUE TRUE TRUE
GB demand flexibility service savingSessions
Analysis of GB electricity demand and twitter data
1 Introduction
UK ‘demand flexibility service’ DFS experiments/trials/‘live’ events by NG-ESO and retailers such as @OctopusEnergy
- DFS began on 1 November 2022.
- The service will run until March 2023.
Relevant media articles:
- Monday 23rd Jan 2023: https://www.bbc.co.uk/news/business-64367504
- Tuesday 24th Jan 2023: https://www.bbc.co.uk/news/business-64372264
Questions:
Q1: Is there a noticeable ‘dip’ in demand? (Can we even tell?)
Q2: Do tweets tell us anything useful about the events?
2 Code setup
Part of https://github.com/dataknut/savingSessions
3 Data
3.1 Electricity demand and carbon intensity data
Use the half-hourly NG-ESO ‘generation mix’ data as a reasonable proxy for ‘demand’. This also includes a useful estimate of carbon intensity.
Should probably use the demand data instead as it includes embedded wind & solar.
Variables of interest:
- GENERATION is in MW. It has some exclusions, its just an indicator.
- CARBON_INTENSITY is in g CO2/kWh - we’d expect this to be high during DFS events (as we’re trying to avoid the use of high carbon & expensive generation?)
3.2 Twitter data
Use rtweet to collect tweets matching:
#savingSessionor#savingSessions(not case sensitive) - this seems to be the preferred Octopus hashtag#DemandFlexibilityService(not case sensitive) - as used by NG-ESO, appears less frequently used
We keep these datasets distinct as not all Octopus DFS events were also wider NG-ESO ‘live’ events.
We do NOT store the tweets in the repo for both ethical and practical reasons…
Note also that we may not be collecting the complete dataset of hashtagged tweets due to the intricacies of the twitter API.
It is important to note that both datasets are therefore likely to be partial.
Figure 1 shows the cumulative number of #savingSession tweets collected and marks the relevant events. Labels are likely to overlap where events are close together.
Figure 2 shows the cumulative number of #DemandFlexibilityService tweets collected and marks the relevant events. Labels are likely to overlap where events are close together.
NB: data collection for this hashtag only started on 23rd January - it looks like twitter’s search API does not return tweets more than x days old. People were tweeting this hashtag earlier than January…
4 Analysis
Q1: Demand reduction
Simple comparison of the ‘demand response session’ day half-hourly demand with the mean half-hourly demand for the previous n days.
All sorts of caveats apply:
- not all of these sessions were national ‘live’ events, some were trials by a single retailer (Octopus)
- we may not be comparing like with like (cold vs warm, weekday vs weekend etc)
- we can’t tell from this analysis what demand would have looked like without the demand response intervention. That would require all the retailers to aggregate the ‘savings’ they estimate… or a randomised control trial
Q2: Twitter sentiment analysis
Inspired by https://www.tidytextmining.com/sentiment.html (sentiment analysis) and https://towardsdatascience.com/create-a-word-cloud-with-r-bde3e7422e8a (word clouds)
Extract the tweets for the day of each DFS event and the day after.
Remove stop words (to, the, and, a, for, etc) & profanity etc from the tweets and sentiment them.
In each case we report the number of negative and positive codings (according to tidytext::get_sentiments("bing")) for the unique words. This will add up to the Number of unique words by sentiment. We then report the total frequency of words that are negative or positive (which will add up to the total number of words).
Remember the size of the (positive/negative) words in the word cloud is relative to the count of all (positive/negative) words in the wordcloud, not the total word count.
4.1 Session 1 (Octopus: Tue 15 Nov 2022 17:00)
- 2022-11-15 17:00:00 to
- 2022-11-15 18:00:00
(Not all retailers may have taken part in this one)
According to https://energycentral.com/news/octopus-energy-provide-108mw-grid-flexibility-first-%E2%80%98saving-session%E2%80%99:
“Over 200,000 households reduced their energy demand by 108MW collectively, the same as a gas power station can generate in an hour. If replicated by all UK energy suppliers, this would be over 1GW.”
But https://www.current-news.co.uk/news/octopus-energy-customers-provide-108mw-of-flexibility-during-first-saving-session quotes Octopus as actually saying:
“if the program was scaled to all electric smart meter customers with all supplier[s] in Great Britain, it could create over 1GW of flexible energy load”
The following plot compares GB electricity demand during the DFS event with mean GB electricity demand over the last n days.
hms | Comparison mean | Saving session | diff | pc_diff |
|---|---|---|---|---|
17:00:00 | 40,226.7 | 42,925.0 | 2,698.3 | 6.7 |
17:30:00 | 40,238.7 | 42,736.0 | 2,497.3 | 6.2 |
Confounding factors:
- recent temperature trends
- recent holidays
- etc
Note that the comparison plot/table shows the difference between the DFS event day and the previous similar days. This does not show what the event day energy use would have been in the absence of the intervention - we’d need a randomised control on the day for that (or a sophisticated demand model as the counterfactual).
Octopus customers’ 108 MW was 0.252 % of mean GB generation (42.925 GW) over the hour of the session (see NG-ESO data above).
108 MW over 200,000 customers is 5.4^{-4} MW per household or, more sensibly, 0.54 kW per household. A pretty reasonable reduction.
But if 200,000 produced a 108 MW power reduction (0.54 kW per household ), the 1 GW estimate implies we only have 1,851,852 GB households with a smart meter. Really? Or did they mean all suppliers taking part in the scheme?
Anyway, the latest data has 14 million domestic eletricity smart meters as of Q1 2022 - 45% of all domestic meters.
So:
- If all ~28 million households in the UK did this it would be 1.512^{7} kW. That’s 15.12 GW…
- If we reduce ambition a bit and assume that only the 14 million domestic GB customers with a smart meter as of Q2 2022 did this then that would provide 7.56 GW. Which would be 17.61 % of generation. Not bad aye?
The trouble is… these are octopus customers self-selecting into a trial. You just have to read the tweets to see that there are households with installed batteries, pausable heat pumps in well-insulated homes and EV V2G. At what point will we all be similarly equipped?
4.1.1 Carbon intensity
And just to keep in mind the carbon context:
Turning to hashtags, we only have #savingSessions tweets for this event.
4.1.2 #savingSessions tweets
Number of tweets: 642
Positive words…
Joining, by = "word"
Joining, by = "word"
Negative words…
Joining, by = "word"
Joining, by = "word"
Total word and unique word counts…
| Var1 | Freq |
|---|---|
| negative | 87 |
| positive | 104 |
| sentiment | freq |
|---|---|
| negative | 203 |
| positive | 349 |
4.2 Session 2 (Octopus: Tue 22 Nov 2022 17:30)
- 2022-11-22 17:30:00 to
- 2022-11-22 18:30:00
(Not all retailers may have taken part in this one)
The same caveats about the comparisons apply here…
hms | Comparison mean | Saving session | diff | pc_diff |
|---|---|---|---|---|
17:30:00 | 41,706.9 | 41,355.0 | -351.9 | -0.8 |
18:00:00 | 41,339.7 | 40,784.0 | -555.7 | -1.3 |
4.2.1 Carbon intensity
And just to keep in mind the carbon context:
Turning to hashtags, we only have #savingSessions tweets for this event.
4.2.2 #savingSessions tweets
Number of tweets: 729
Positive words…
Joining, by = "word"
Joining, by = "word"
Negative words…
Joining, by = "word"
Joining, by = "word"
Total word and unique word counts…
| Var1 | Freq |
|---|---|
| negative | 107 |
| positive | 106 |
| sentiment | freq |
|---|---|
| negative | 197 |
| positive | 410 |
4.3 Session 3 (Octopus: Wed 30 Nov 2022 17:30)
- 2022-12-01 17:00:00 to
- 2022-12-01 18:00:00
(Not all retailers may have taken part in this one)
The same caveats about the comparisons apply here…
hms | Comparison mean | Saving session | diff | pc_diff |
|---|---|---|---|---|
17:30:00 | 42,687.9 | 41,488.0 | -1,199.9 | -2.8 |
18:00:00 | 42,102.9 | 41,011.0 | -1,091.9 | -2.6 |
4.3.1 Carbon intensity
And just to keep in mind the carbon context:
Turning to hashtags, we only have #savingSessions tweets for this event.
4.3.2 #savingSessions tweets
Number of tweets: 728
Positive words…
Joining, by = "word"
Joining, by = "word"
Negative words…
Joining, by = "word"
Joining, by = "word"
Total word and unique word counts…
| Var1 | Freq |
|---|---|
| negative | 77 |
| positive | 103 |
| sentiment | freq |
|---|---|
| negative | 206 |
| positive | 490 |
4.4 Session 4 (Octopus: Thu 01 Dec 2022 17:00)
- 2022-12-01 17:00:00 to
- 2022-12-01 18:00:00
(Not all retailers may have taken part in this one)
The same caveats about the comparisons apply here…
hms | Comparison mean | Saving session | diff | pc_diff |
|---|---|---|---|---|
17:00:00 | 42,751.9 | 42,065.0 | -686.9 | -1.6 |
17:30:00 | 42,693.6 | 42,194.0 | -499.6 | -1.2 |
4.4.1 Carbon intensity
And just to keep in mind the carbon context:
Turning to hashtags, we only have #savingSessions tweets for this event.
4.4.2 #savingSessions tweets
Number of tweets: 305
Positive words…
Joining, by = "word"
Joining, by = "word"
Negative words…
Joining, by = "word"
Joining, by = "word"
Total word and unique word counts…
| Var1 | Freq |
|---|---|
| negative | 40 |
| positive | 62 |
| sentiment | freq |
|---|---|
| negative | 83 |
| positive | 189 |
4.5 Session 5 (Octopus: Mon 12 Dec 2022 17:00)
- 2022-12-12 17:00:00 to
- 2022-12-12 18:00:00
(Not all retailers may have taken part in this one)
The same caveats about the comparisons apply here…
hms | Comparison mean | Saving session | diff | pc_diff |
|---|---|---|---|---|
17:00:00 | 43,849.1 | 47,821.0 | 3,971.9 | 9.1 |
17:30:00 | 43,871.3 | 48,470.0 | 4,598.7 | 10.5 |
4.5.1 Carbon intensity
And just to keep in mind the carbon context:
Turning to hashtags, we only have #savingSessions tweets for this event.
4.5.2 #savingSessions tweets
Number of tweets: 585
Positive words…
Joining, by = "word"
Joining, by = "word"
Negative words…
Joining, by = "word"
Joining, by = "word"
Total word and unique word counts…
| Var1 | Freq |
|---|---|
| negative | 41 |
| positive | 73 |
| sentiment | freq |
|---|---|
| negative | 138 |
| positive | 362 |
4.6 Session 6 (Octopus: Thu 19 Jan 2023 09:00)
- 2023-01-19 09:00:00 to
- 2023-01-19 10:00:00
(Not all retailers may have taken part in this one)
The same caveats about the comparisons apply here…
hms | Comparison mean | Saving session | diff | pc_diff |
|---|---|---|---|---|
09:00:00 | 39,729.3 | 45,823.0 | 6,093.7 | 15.3 |
09:30:00 | 40,463.5 | 46,739.0 | 6,275.5 | 15.5 |
4.6.1 Carbon intensity
And just to keep in mind the carbon context:
Turning to hashtags, we only have #savingSessions tweets for this event.
4.6.2 #savingSessions tweets
Number of tweets: 248
Positive words…
Joining, by = "word"
Joining, by = "word"
Negative words…
Joining, by = "word"
Joining, by = "word"
Total word and unique word counts…
| Var1 | Freq |
|---|---|
| negative | 22 |
| positive | 40 |
| sentiment | freq |
|---|---|
| negative | 37 |
| positive | 85 |
4.7 Session 7 (NG-ESO ‘live’: Mon 23 Jan 2023 17:00)
This was a more generalised call for DFS:
- 2023-01-23 17:00:00 to
- 2023-01-23 18:00:00
The same caveats about the comparisons apply here…
hms | Comparison mean | Saving session | diff | pc_diff |
|---|---|---|---|---|
17:00:00 | 43,930.7 | 46,086.0 | 2,155.3 | 4.9 |
17:30:00 | 44,034.9 | 45,684.0 | 1,649.1 | 3.7 |
4.7.1 Carbon intensity
And just to keep in mind the carbon context:
Turning to hashtags, we have both #savingSessions and #demandFlexibilityService tweets for this event.
4.7.2 #savingSessions tweets
Number of tweets: 593
Positive words…
Joining, by = "word"
Joining, by = "word"
Negative words…
Joining, by = "word"
Joining, by = "word"
Total word and unique word counts…
| Var1 | Freq |
|---|---|
| negative | 88 |
| positive | 90 |
| sentiment | freq |
|---|---|
| negative | 179 |
| positive | 317 |
4.7.3 #demandFlexibilityService tweets
Number of tweets: 166
Positive words…
Joining, by = "word"
Joining, by = "word"
Negative words…
Joining, by = "word"
Joining, by = "word"
Total word and unique word counts…
| Var1 | Freq |
|---|---|
| negative | 23 |
| positive | 32 |
| sentiment | freq |
|---|---|
| negative | 68 |
| positive | 108 |
4.8 Session 8 (NG-ESO ‘live’: Tue 24 Jan 2023 16:30)
This was another generalised call for DFS:
- 2023-01-24 16:30:00 to
- 2023-01-24 18:00:00
The same caveats about the comparisons apply here…
Also this ecent followed very soon after the last one so this will affect the comparison ‘baseline’ (previous n days).
hms | Comparison mean | Saving session | diff | pc_diff |
|---|---|---|---|---|
16:30:00 | 43,409.2 | 44,975.0 | 1,565.8 | 3.6 |
17:00:00 | 44,395.9 | 45,275.0 | 879.1 | 2.0 |
17:30:00 | 44,466.2 | 44,814.0 | 347.8 | 0.8 |
4.8.1 Carbon intensity
And just to keep in mind the carbon context:
Turning to hashtags, we have both #savingSessions and #demandFlexibilityService tweets for this event.
4.8.2 #savingSessions tweets
Number of tweets: 462
Positive words…
Joining, by = "word"
Joining, by = "word"
Negative words…
Joining, by = "word"
Joining, by = "word"
Total word and unique word counts…
| Var1 | Freq |
|---|---|
| negative | 77 |
| positive | 68 |
| sentiment | freq |
|---|---|
| negative | 140 |
| positive | 217 |
4.8.3 #demandFlexibilityService tweets
Number of tweets: 58
Positive words…
Joining, by = "word"
Joining, by = "word"
Negative words…
Joining, by = "word"
Joining, by = "word"
Total word and unique word counts…
| Var1 | Freq |
|---|---|
| negative | 9 |
| positive | 19 |
| sentiment | freq |
|---|---|
| negative | 36 |
| positive | 54 |
5 Future work
- more trial days
- use 5 days before/after as comparators? Confounded by events close together…